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ABSTRACT 

This paper describes changing the way a business statistics course is taught through the use of 
just-in-time teaching methods. Implementing this method allowed for more time in the class to be 
spent focused on problem solving, resulting in students being able to handle more difficult 
problems. Students ’perceptions of the just-in-time assignments were overall positive. 
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INTRODUCTION 



- e all want our students to come to class prepared. However, we also know that a significant number 
of our students will come to class completely unprepared. This situation can be particularly difficult 
to deal with in a business statistics course. I often find that I have a wide variety of prior experience 
and comfort with quantitative skills in my business statistics course. Lack of pre-class preparation, particularly by 
struggling students can make for a difficult teaching situation. I have over the years attempted to design the course 
for the prepared student while continually reminding students of the need to come to class prepared. I have also 
attempted to design the course for the unprepared student. Neither of these cases led to particularly good teaching 
experiences. In this paper, I describe using the active learning technique known as just-in-time teaching (JiTT) to 
transform the way I taught my business statistics course. I examine both student performance in the course and 
student attitudes towards the teaching method. 


SUPPORT FOR USING JiTT 


The basic process of JiTT involves assigning questions for students to complete and submit (often through 
a course management system) several hours or perhaps the day before the class meets. Then the instructor uses the 
responses from the students to shape in class activities to address learning gaps displayed in the pre-class 
assignments. It is this second step by the instructor that distinguishes JiTT from merely giving students a quiz on 
the readings. A number of studies have been done to show the effectiveness of JiTT as a teaching strategy. A nice 
review of these findings can be found in Simkins and Maier (2004). First and foremost amongst the benefits of JiTT 
is that the students come to class prepared. A well designed JiTT question helps the student to focus on particular 
content rather than passively reading the textbook. Other benefits of JiTT come from the process of using specific 
student responses (anonymously) to generate discussion in the class. Seeing their own responses used in class 
creates some ownership in the learning process. It helps both the students and the instructor understand where 
misconceptions are coming from. 

Simkins and Maier (2004) also report on the effect of JiTT on learning. In their study students were 
randomly assigned to two different groups. The groups were assigned either JiTT exercises or alternative exercises 
at different points during the course. After controlling for factors such as GPA and SAT scores, they were only able 
to show a weak (not statistically significant) positive effect from JiTT. However, as the authors note, their process 
of having both groups complete JiTT exercises during different parts of the course may have affected their results. 
It is possible that those students that were introduced to JiTT early in the term may have changed their study habits 
as a result. When measuring the effect on student learning early in the course (when only one group had completed 
JiTT exercises to date) there was a positive JiTT effect at the 10% level of significance. Authors in other fields have 
also tested the effectiveness of JiTT. For example, Slunt and Giancarlo (2004) compared the use of Concept Test 
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and JiTT for Chemistry students. They found that both of these active learning techniques improved student 
performance. However, they also found that JiTT did more to improve student participation than the Concept Test. 
For other examples, see Marrs and Novak (2004) for an assessment of JiTT in Biology or Grove (2002) for 
Geoscience. 

PROCESS OF USING JiTT 

I introduced JiTT into my business statistics course in the summer session of 2011. Although the course is 
designed for business and economics majors, there are a number of other majors that may use the course to fulfill 
their statistics requirement. As a consequence, there is generally a wide variety of backgrounds in and comfort with 
quantitative areas. Previously the course had been taught with a mixture of lecture and in-class problem solving. 
While the in-class problem solving seemed to be the most valuable activity in the course, I found that I was not able 
to devote enough time to this activity as too much time was devoted to lecture. I decided to try implementing JiTT 
to see if improved student preparation could lead to allowing more time for in class problem solving. 

The process of using JiTT 1 begins with designing good questions. One important aspect of the questions 
was that they needed to be complex. I found that the questions that worked best were the ones that no one was able 
to get completely correct. That way all of the students could be pushed in class to think beyond what they had done 
before the class. 

The second step in the process for the instructor is to review the student responses to the pre-class questions 
and to then design a portion of the class meeting time around the answers given by the students. My class met once 
a week at night and students had to turn in the pre-class assignments the morning of the class. As I reviewed the 
student responses, I would categorize them by the particular approach taken by the students. Often a clear 
progression would emerge from these categories. I was generally able to see a path to take to describe how to 
approach solving a complex problem by looking at where students got stuck in the process on their pre-class 
assignments. I then chose a sample response to show to the class (anonymously) from each of the categories. Over 
the course of the term I tried to make sure I was selecting responses from as many different students as was possible. 

Finally the process ends with the classroom activity. As much as possible I asked students to try to identify 
where there were gaps or wrong turns in the sample answers. We did this work either in small groups or as whole 
class discussions. This discussion would provide a spring board for a mini lecture on the concepts we were 
discussing that evening. 

1 did choose to assign grades to the pre-class assignments. I was concerned that students would not take the 
pre-class assignments seriously if they did not affect their grade. However, I did not wish to overly penalize the 
students for failing to completely solve the pre-class assignments, as I was trying to design questions that would be 
too difficult for the students to completely solve before class. I chose to make the pre-class assignments in total 
account for about 9% of their course grade. I also gave students the following rubric that I used to grade the pre- 
class assignments. 


Grade 

Points 

Scoring Criteria 

A 

9-10 

Student answers all questions correctly and completely. Student incorporates information from the text 
into the answer. 

B 

8-8.5 

Student uses statistical concepts to answer the questions. Student does not use all appropriate statistical 
concepts. Answers may be partially correct but still not complete. 

C 

7-7.5 

Student uses statistical concepts to answer the questions. Student does not use appropriate statistical 
concepts. Answers are incorrect. 

D 

6-6.5 

Student tries to answer the questions, but does not show evidence of using information from the text. 
Answers are incorrect. 

F 

0-5.5 

Student is not able to answer the questions. 


1 Starting Point: Teaching and Learning Economics (see references for URL) is a very good resource for help in getting starting 
with using JiTT. 
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As an instructor I was very pleased with the implementation of JiTT into my business statistics course. I 
found that my students came into the room with a much better base of knowledge than I had seen in all previous 
times I had taught the course. I also found that the students came to each class with a more equal footing. Although 
there were certainly still differences in ability and comfort level, the struggling students were much better equipped 
to learn each night. I found I was able to significantly increase the amount of time I spent in class doing problem 
solving. As a matter of fact, the problem solving became the focus of the class each night, rather than the piece I did 
my best to fit in. I also found it a very refreshing experience to completely rethink my approach to the material 
before each class. Although I still carried the notes I had used for years to class with me each night, 1 found I almost 
never used these notes. Instead I was engaged in a much more interactive process with the students on how to apply 
statistical concepts. In the past I had spent much more time simply trying to explain what the statistical concepts 
were. Unfortunately, I cannot document any improvement in student grades. The distribution of course grades in 
this class looked very similar to distributions from past years. However, I did incorporate more complex questions 
into post-class homework assignments and exams than I had in the past. So, it seems to me as if the students 
completed the class with a better understanding of statistics despite the lack of change in the grade distribution. 

In the last class meeting, I asked the students to complete a survey on the instructional methods used in the 
course. A copy of the survey questions in included in Appendix A. In the first question, I asked students to rate the 
usefulness of various components of the course on a 5 point scale with 1 being not at all useful and 5 being very 
useful. The modal response was 5 for discussing pre-class assignments in class, completing post-class homework 
assignments, discussing graded post-class homework assignments in class, lecture and in-class problem solving. 
The modal response was 3 for completing pre-class assignments, reading the textbook and solving odd number 
problems from the textbook. I found it interesting that students seemed to find discussing the pre-class assignments 
in class more useful than actually completing the pre-class assignments. (See Appendix B for a distribution of 
responses for these two items.) Looking at individual responses, in most cases students gave a higher usefulness 
rating for discussing the pre-class assignment than they did for completing the pre-class assignments. It seems that 
students were quite engaged in the process of discussing the pre-class questions in class, but did not make the 
connection to the need to have completed the problems before class in order to have such a discussion. It is also 
noteworthy that reading the textbook had a lower usefulness rating from the students. Again there seemed to be a 
disconnect for the students. The richness of the discussion of the pre-class questions came about because the 
students had spent time reading the textbook and attempting to complete the pre-class questions prior to class. I 
have had students tell me informally that they felt like they were wasting time struggling to understand the material 
in the textbook to complete the pre-class assignment, because it all seemed so easy once they heard me explain it in 
class. I continually work to explain to students that things seem easy in class because they have spent the time 
struggling with the material on their own first, but it seems I have not yet fixed this problem. 

In the survey, I also asked students to tell me what they thought worked best about the pre-class 
assignments, what recommendations they had to improve the use of pre-class assignments, and whether or not they 
would recommend continuing to use the pre-class assignments. The reaction to the pre-class assignments was 
mixed, but 53% of the students would recommend continuing to use pre-class assignments in this course. Most 
frequently the positive comments from students were in the area of preparedness. Students did seem to recognize 
that working on the pre-class assignments forced them to be prepared for class, though as noted above that did not 
necessarily translate to perceived usefulness of completing the pre-class assignments. The following student 
comments illustrate this theme: 

The assignment helped me to anticipate what I would learn in class. 

I really like the Fact that you gave us pre-class problems. It made the next class more interesting & I could follow 
along and ask Questions where the book made me confused. 

Forced me to read the chapters and helped me to understand the material being presented. 


I had a heads up as to what to expect for the week. 
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Although pre-class’s are naturally more confusing than post-class, it seemed to be a useful way to start to 
understand the new material, and going over them in class solidified or clarified what we understood on our own. 

It is good prep for class and motivation to stay ahead. 

It is a great way to stay on top for the class. And it makes you read your text book. Without it I don't think I would 
have even opened it. Wonderful Idea! :) 

However, there were also a number of negative comments about the pre-class questions. These responses tended to 
fall into themes of frustration, concern about the effect on their grade, or feeling that working from the textbook did 
not match their learning style. The following student comments illustrate these themes: 

I didn't like these at all. It mostly made me stressed out that I had to learn something on my own and I am kind of a 
perfectionist so I would spend hours trying to figure some out. 

I found it very hard to self teach myself how to solve the problems since reading and comprehension is a difficult 
learning style for me with math. 

They really stressed me out. . . I spent far too much time trying to figure these out. 

The way I learn is through seeing others do and then modeling their work. This is why the post-homework 
assignment works best for me. 

I also asked students to tell me how much time they spent on the pre-class assignments. Most students 
indicated they spent around 2 hours each week on these assignments. This time was about half the amount of time 
students indicated they spent on the post-class assignments (traditional homework assignments). Despite comments 
to the contrary, the pre-class assignments did not seem to take up too much of the students’ time. 

Finally I asked students to tell me about the number of points assigned to the pre-class questions. Students 
could indicate ‘Not enough points,’ ‘About the right amount of points,’ or ‘Too many points.’ The modal response 
(69%) was ‘About the right amount of points.’ The remaining students indicated ‘Too many points.’ Despite the 
high percentage of students indicating they thought the amount of points assigned was fine, I believe lowering the 
number of points in the future could be beneficial. I think it might reduce some of the stress for students 
surrounding completing the pre-class assignments. However, it will be interesting to see if the amount of time 
students spent on the pre-class assignments is negatively affected by reducing the number of points assigned to 
them. 

CONCLUSION 

Despite the concerns expressed by some of my students, I consider the implementation of JiTT into my 
business statistics course a success. For me, the introduction of JiTT completely transformed the way I thought 
about the course. After teaching numerous sections of this course over the past several years, I felt completely 
renewed and refreshed in my teaching simply by being forced to react to the responses I received from students. 
However, there do remain improvements to be made the next time around. To reduce the anxiety surrounding 
completing the pre-class assignments, I will reduce the number of points for these. I will likely also alter the 
grading rubric to place greater emphasis on application of material from the reading, rather than correct application 
of the material from the reading. Making this change should help me to engage the students in a discussion about 
learning from mistakes and building our knowledge through a process. One final change I will make is to give each 
of the students individualized feedback on their pre-class assignments. I had made the assumption that the in-class 
discussion of the pre-class assignments would provide the students with enough feedback. However, comments on 
the student survey lead me to believe this process was not sufficient. 
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Your participation in this survey is voluntary. If you do not wish to participate, please leave this sheet blank. If 
there are any questions you prefer not to answer, you may leave those blank. 

1) How useful were each of the following components of the course in helping you to learn the material? Please 
rate the usefulness of each of these components of the course on a scale of 1 to 5, with 1 being not at all useful 
and 5 being very useful. 

_Completing pre-class assignments 

_Discussing pre-class assignments in class 

_Completing post-class homework assignments 

_Discussing graded post-class homework assignments in class 

_Lecture 

_In-class problem solving 

_Reading the textbook 

_Solving odd number problems from the textbook 

2) On average, how much time did you spend each week completing the pre-class assignments? 


3) On average, how much time did you spend each week completing the post-class homework assignments? 


4) How do you feel about the number of points possible on the pre-class assignments? Please circle your response. 

Not enough points About the right amount of points Too many points 

5) How do you feel about the number of points possible on the post-class homework assignments? Please circle 

your response. 

Not enough points About the right amount of points Too many points 

6) What worked best about our process of using the pre-class assignments to discuss new material? 

7) What recommendations do you have to improve the process of using the pre-class assignments to discuss new 
material? 

8) Overall, would you recommend continuing to use pre-class assignments in this course? Why or why not? 
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